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Period for Reply 


A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS. 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even If timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to communication(s) filed on 12 September 2005 . 
2a)\3 This action is FINAL. 2b)|3 This action is non-final. 

3) 0 Since this application is in condition for allowance except for fonrial matters, prosecution as to the merits is 

closed in accordance with the practice under £x parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) |E Claim(s) 1-22 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) \3 Claim(s) is/are allowed. 

6) 13 Claim(s) 1-22 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: 3)0 accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing{s) be held in abeyance. See 37 CFR 185(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or fomri PTO-152. 

Priority under 35 U.S.C. § 119 

12)0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f), 
a)n All b)n Some * 0)0 None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Claim Rejections - 35 USC§103 
The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue, 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

1 . Claims 1-22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Garg et al, 
"Frame-dependent multi-stream reliability indicators for audio-visual speech recognition/' 
Proceedings of Intemational Conference on Acoustics, Speech and Signal Processing, ICASSP 
2003, vol. 1, April 2003, pages 24-27 in view of Masai et al (US Patent Application Publication 
2003/0177005). 

2. Regarding claim 1, Garg teaches a method for audio-visual speech recognition 
comprising: providing an acoustic-only data model and an acoustic-visual data model (pages 
24-26; section 2, entitled "The Multi-Stream HMM"; section 3, entitled "Stream Reliability 
Indicators"; section 4, entitled "Reliability Based Stream Exponents."); and decoding at least a 
portion of an input spoken utterance using selected data models (pages 24-26; section 2, entitled 
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"The Multi-Stream HMM"; section 3, entitled "Stream Reliability Indicators"; section 4, 
entitled "Reliability Based Stream Exponents"; Tables 1-2). Garg does not specifically teach a 
data model is selected based on a condition associated with the environment of the speaker. 
However, selecting an optimum data model for performing recognition based on environmental 
conditions so as to improve recognition accuracy and performance was well known in the art of 
speech recognition. Masai discloses (paragraph 75) a method and device for producing acoustic 
models for recognition and specifically teaches the speech recognition unit recognizes the 
speech data and convert them into text data in accordance with the environment information of 
the time when the speech data are uttered, the acoustic model for recognition selection unit 
selects the acoustic model for recognition according to the environment information and 
converts the speech data into text data by using the selected acoustic model for recognition. 

It would have been obvious to one of ordinary skill at the time of the invention to modify 
the system of Garg to allow for the selection of the most optimum data model, as suggested by 
Masai, for the purpose of improving recognition accuracy and performance of the speech 
recognizer, as was well known in the art. 

Regarding claim 2, Garg and Masai teach storing the acoustic-only data model and the 
acoustic-visual data model in memory such that mbdel selection is made by shifting one or more 
pointers to one or more memory locations where the selected model is located (Page 26-27, 
section 5, "Database and Experiments"). 

Regarding claim 3, Grarg and Masai teach model selection is based on a likelihood ratio 
test (pages 24-26; section 2, entitled "The Multi-Stream HMM"; section 3, entitled "Stream 
Reliability Indicators"; section 4, entitled "Reliability Based Stream Exponents"). 
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Regarding claim 4, Garg and Masai teach model selection comprises selecting the 
acoustic-only data model when a result of the likelihood test is not greater than a threshold value 
(pages 24-26; section 2, entitled "The Multi-Stream HMM"; section 3, entitled "Stream 
Reliability Indicators"; section 4, entitled "Reliability Based Stream Exponents"). 

Regarding claim 5, Garg and Masai teach the model selection step comprises selecting 
the acoustic- visual data mode when a result of the likelihood test is not less than a threshold 
(pages 24-26; section 2, entitled "The Multi-Stream HMM"; section 3, entitled "Stream 
ReUability Indicators"; section 4, entitled "Reliability Based Stream Exponents"). 

Regarding claim 6, Garg and Masai teach the threshold value is based on a cost 
associated with a recognition error (Tables 1 and 2; section 3, "Stream Reliability Indicators). 

Regarding claim 7, Garg and Masai teach the likelihood ratio test is based on one or more 
observations of a given visual feature (Tables 1 and 2; section 3, "Stream Reliability Indicators). 

Regarding claim 8, Garg and Masai teach the given visual feature is associated with the 
mouth region of a speaker of the input utterance (Page 26-27, section 5, "Database and 
Experiments"). 

Regarding claim 9, Garg and Masai teach the model selection is performed at a rate 
substantially equivalent to an observation rate associated with the audio-visual speech 
recognition system (Page 26-27, section 5, "Database and Experiments"). 
3. Regarding claims 10-22; claims 10-22 are similar in scope and content to method claims 
1-9 and are therefore rejected under similar rationale. 
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Response to Arguments 

4. Applicant's arguments with respect to claims 1-22 have been considered but are moot in 
view of the new ground(s) of rejection. 

5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Angela A. Armstrong whose telephone number is 571-272-7598. 
The examiner can normally be reached on Monday-Thursday 1 1:30-8:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on 571-272-7602. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or PubUc PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Angela A Armstrong 
Primary Examiner 
Art Unit 2654 
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December 27, 2005 


